Neural Information Processing Systems http://nips.cc/

Neural Information Processing Systems http://nips.cc/

ZIP Lab, Monash University, Australia We organize our supplementary material as follows: In section A, we provide a comprehensive explanation of extending PTQD to DDIM [10]. In section B, we show the statistical analysis of quantization noise. In section D, we provide additional visualization results on ImageNet and LSUN dataset. We first perform statistical tests to verify if the residual quantization noise adheres to a Gaussian distribution. This test is based on D'Agostino and Pearson's In Figure B, we present the variance of the residual uncorrelated quantization noise.

Diffusion models have recently dominated image synthesis and other related generative tasks. However, the iterative denoising process is expensive in computations at inference time, making diffusion models less practical for low-latency and scalable real-world applications.

Diffusion models have recently dominated image synthesis and other related generative tasks. However, the iterative denoising process is expensive in computations at inference time, making diffusion models less practical for low-latency and scalable real-world applications. Post-training quantization of diffusion models can significantly reduce the model size and accelerate the sampling process without requiring any re-training. Nonetheless, applying existing post-training quantization methods directly to low-bit diffusion models can significantly impair the quality of generated samples. Specifically, for each denoising step, quantization noise leads to deviations in the estimated mean and mismatches with the predetermined variance schedule. Moreover, as the sampling process proceeds, the quantization noise may accumulate, resulting in a low signal-to-noise ratio (SNR) during the later denoising steps. To address these challenges, we propose a unified formulation for the quantization noise and diffusion perturbed noise in the quantized denoising process.

Quantized neural networks can be viewed as a chain of noisy channels, where rounding in each layer reduces capacity as bit-width shrinks; the floating-point (FP) checkpoint sets the maximum input rate. We track capacity dynamics as the average bit-width decreases and identify resulting quantization bottlenecks by casting fine-tuning as a smooth, constrained optimization problem. Our approach employs a fully differentiable Straight-Through Estimator (STE) with learnable bit-width, noise scale and clamp bounds, and enforces a target bit-width via an exterior-point penalty; mild metric smoothing (via distillation) stabilizes training. Despite its simplicity, the method attains competitive accuracy down to the extreme W1A1 setting while retaining the efficiency of STE.

We propose QeRL, a Quantization-enhanced Reinforcement Learning framework for large language models (LLMs). While RL is essential for LLMs' reasoning capabilities, it is resource-intensive, requiring substantial GPU memory and long rollout durations. Beyond efficiency, our findings show that quantization noise increases policy entropy, enhancing exploration, and enabling the discovery of better strategies during RL. To further optimize exploration, QeRL introduces an Adaptive Quantization Noise (AQN) mechanism, which dynamically adjusts noise during training. Experiments demonstrate that QeRL delivers over 1.5 speedup in the rollout phase. Moreover, this is the first framework to enable RL training of a 32B LLM on a single H100 80GB GPU, while delivering overall speedups for RL training. It also achieves faster reward growth and higher final accuracy than 16-bit LoRA and QLoRA, while matching the performance of full-parameter fine-tuning on mathematical benchmarks such as GSM8K (90.8%) and MA TH 500 (77.4%) in the 7B model. These results establish QeRL as an efficient and effective framework for RL training in LLMs.Figure 1: Rollout speedup and accuracy of QeRL on Qwen2.5-7B-Instruct. QeRL achieves faster RL rollout and end-to-end training speeds (batch=8), while delivering performance superior to vanilla LoRA and QLoRA, also comparable to full-parameter RL on mathematical benchmarks. The ability to perform multi-step reasoning is critical for large language models (LLMs) to handle complex tasks, from theoretical problem solving to practical decision making (Sui et al., 2025; Xu et al., 2025; Chu et al., 2025; Y ang et al., 2021). Supervised fine-tuning (SFT) is a common method to improve reasoning by training models to replicate explicit reasoning steps (Huang et al., 2024d; Min et al., 2024). In contrast, reinforcement learning (RL) uses verifiable reward signals to support adaptive learning, allowing models to explore diverse reasoning traces and identify more robust solutions (Lambert et al., 2024; DeepSeek-AI, 2025; Chen et al., 2025a). 1 AQN dynamically adjusts quantization noise with an exponential scheduler, enhancing exploration.

ZIP Lab, Monash University, Australia We organize our supplementary material as follows: In section A, we provide a comprehensive explanation of extending PTQD to DDIM [10]. In section B, we show the statistical analysis of quantization noise. In section D, we provide additional visualization results on ImageNet and LSUN dataset. We first perform statistical tests to verify if the residual quantization noise adheres to a Gaussian distribution. This test is based on D'Agostino and Pearson's In Figure B, we present the variance of the residual uncorrelated quantization noise.